fix(enterprise): fail closed on unsupported anonymization config; reject non-finite trust weights - #358
Open
kevincostner17 wants to merge 1 commit into
Open
fix(enterprise): fail closed on unsupported anonymization config; reject non-finite trust weights#358kevincostner17 wants to merge 1 commit into
kevincostner17 wants to merge 1 commit into
Conversation
…non-finite trust weights EnterpriseConfig.anonymization was accepted and type-checked but no pipeline stage ever read it, so clean_enterprise returned raw PII with no mask or privacy report and no warning. clean_enterprise now raises ValueError when the field is non-empty and points callers at masking= (MaskingRule) or privacy= (PIIDetectionConfig with enable_privacy_detection=True). EnterpriseConfig still constructs with the field, and it is documented as unsupported. The CLI builds its config without this field and goes through clean_enterprise, so it inherits the guard (reported as a one-line error, exit 1). A privacy control that silently does nothing is worse than an explicit error; the guard can be removed once the field is actually implemented. TrustScoreWeights only checked `value < 0` and a positive sum, both of which are False for NaN, and +inf passed too; either produced a NaN trust score graded F. Each weight must now be finite (math.isfinite), otherwise ValueError. Closes #247 Closes #277
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
FreshData benchmark report —
|
| fixture | n_rows | n_cols | p50 s | p95 s | peak MB | repair % | false-repair % | preserve % | trust | monotonic | export % |
|---|
Authored-code reduction (Metric 6)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#247:
EnterpriseConfig.anonymizationis accepted but never applied.EnterpriseConfig.__post_init__type-checks the field, but no stage ofclean_enterprisereads it. Masking is driven only bymaskingandprivacy. A caller who setanonymization=(AnonymizationConfig(strategy="redact"),)got raw PII back, withmask_report=None,privacy_report=Noneand no warning.clean_enterpriseraisesValueErrorwhenanonymizationis non-empty. The message points tomasking=(MaskingRule) orprivacy=(PIIDetectionConfigwithenable_privacy_detection=True).EnterpriseConfigstill constructs with the field, so existing config objects keep loading.EnterpriseConfig,AnonymizationConfigandclean_enterprisedocstrings.enable_maskingis set.clean_enterprise, so they fail the same way. The CLI prints a one-line error and exits 1.anonymizationis implemented, the check comes out and the same configs start working, with no API break.#277:
TrustScoreWeightsaccepts NaN and inf.value < 0andsum <= 0are both false for NaN, and +inf passes both, so the trust score came outnanwith grade F. Each weight must now be finite (math.isfinite), otherwiseValueError.Tests
tests/test_enterprise_config.py:EnterpriseConfig(anonymization=...)still constructs and type-checks.tests/test_enterprise_interface.py:clean_enterpriseraises withenable_maskingboth True and False.cleanpath exits non-zero, prints the error to stderr and writes no output file.Verification
ruff check .: all checks passedmypy src/freshdata: no issues in 202 source filespytest -m "not online and not large", Python 3.12: 4256 passed, 6 skippedpytest -m "not online and not large", Python 3.9 / pandas 1.5: 4252 passed, 10 skippedValueError.Closes #247
Closes #277